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GRANT  # :  N000149510241 

PRINCIPAL  INVESTIGATOR:  Dr.  Todd  R.  Johnson 

INSTITUTION :  The  University  of  Texas  Houston  Health  Science 
Center 

GRANT  TITLE :  A  Hybrid  Learning  Model  of  Tactical  Decision 
Making 

AWARD  PERIOD:  1  Jan  1995  -  15  May  1999. 

OBJECTIVE:  To  develop  a  hybrid  learning  architecture  for 
decision  making  tasks  that  involve  generation,  revision  and 
evaluation  of  explanatory  hypotheses;  to  investigate 
learning  in  the  context  of  the  CIC  (Combat  Information 
Center)  tactical  decision  making  task. 

APPROACH :  We  studied  how  people  acquire  and  use  statistical 
information  in  the  CIC  tactical  decision  making  task.  We 
specifically  investigated  the  acquisition  of  prior  and 
conditional  probabilities,  how  the  order  of  evidence 
affects  probabilistic  decision  making  (i.e.,  order 
effects),  and  how  experience  affects  order  effects.  We 
modeled  the  experimental  results  using  a  hybrid  symbolic- 
connectionist  cognitive  model.  We  used  Soar  as  the  symbolic 
component  and  a  modified  version  of  Echo  as  the 
connectionist  component. 

Accomplishments :  We  completed  several  experiments  examining 
the  acquisition  and  use  of  probabilistic  information  and 
its  relation  to  order  of  data  presentation  effects.  The 
order  effect  occurs  when  individuals  come  to  different 
beliefs  based  only  on  the  order  in  which  evidence  is 
presented  to  them.  We  found  that  individuals  could  acquire 
and  use  probabilities  (as  predicted  by  several  current 
theories) ,  but  that  their  judgments  still  exhibited  order 
effects,  a  result  that  was  inconsistent  with  existing 
models  of  probability  learning  and  use.  We  constructed  a 
symbolic -connect ionist  model,  based  on  Thagard's  Echo,  that 
could  acquire  probabilities  and  exhibit  order  effects 
similar  to  those  found  in  human  subjects.  The  result  is  an 
extended  version  of  Echo  that  is  potentially  applicable  to 
a  wide  range  of  tasks . 
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We  conducted  one  experiment  to  examine  the  effects  of 
different  types  of  training  on  tactical  decision  making.  In 
particular,  we  examined  the  role  of  Cohen's  metacognitive 
training  on  tactical  decisions.  Cohen  had  shown  that  this 
training  enhanced  decision  making  in  the  real-world  CIC 
tactical  decision  making  task  when  used  with  experienced 
decision  makers.  Metacognitive  reasoning  involves  the 
forced  consideration  of  alternative  possibilities.  Our  goal 
was  to  see  if  the  same  training  might  affect  novice 
decision  makers.  We  found  no  effect  of  the  metacognitive 
training  on  the  CIC  decisions.  However,  we  believe  that 
this  has  more  to  do  with  the  paucity  of  knowledge  in  the 
CIC  simulation,  and  not  the  experience  level  of  the 
subjects.  If  novice  subjects  were  given  a  richer  knowledge 
base,  they  might  benefit  from  metacognitive  training. 

We  also  conducted  several  experiments  to  examine  how 
situational  shifts  (such  as  from  a  peaceful  to  a  hostile 
situation)  would  affect  decision  making.  We  found  that  such 
shifts  produced  nonmonotonic  changes  to  decision  making,  in 
which  the  decision  maker  would  rapidly  change  their 
decisions  to  better  match  the  new  situation.  Our  results 
suggest  that  probability  acquisition  is  subject  to  a 
recency  effect,  wherein  problem  solvers  give  more  weight  to 
recent  experience  than  to  distant  experience. 

In  the  past  year,  Hongbin  Wang,  the  graduate  research 
assistant  on  this  project,  successfully  defended  his 
PhD  thesis,  which  was  based  completely  on  this 
project.  The  most  significant  finding  from  this  work 
is  that  order  effects  in  human  belief  revision  (such 
as  recency  and  primacy  induced  by  the  order  in  which 
evidence  is  presented)  disappear  with  experience.  That 
is,  as  subjects  get  more  practice  with  a  belief 
revision  task,  the  order  effects  slowly  disappear  and 
their  beliefs  more  closely  match  the  normative 
probabilities.  We  have  also  produced  a  cognitive  model 
of  belief  revision  that  models  these  results.  This 
model  is  generally  applicable  to  any  belief  revision 
task. 

One  of  the  problems  with  the  CIC  simulation  task  was 
the  lack  of  a  gold  standard  for  each  scenario.  This 
made  it  difficult  to  study  learning  using  CIC  and  to 
compare  the  results  of  different  learning  models.  In 
particular,  there  was  no  way  to  give  clear  feedback  to 
subjects  about  whether  or  not  their  decisions  were 


right  or  wrong.  In  collaboration  with  Sandra  Marshall 
(who  is  funded  under  a  related  ONR  Hybrid  Learning 
grant) ,  we  developed  a  set  of  scenarios  based  on 
expert  decision  making  in  the  original  TADMUS 
(Tactical  Decision  Making  Under  Stress)  scenarios. 

These  scenarios  allowed  us  to  collect  learning  data  on 
novice  subjects  and  to  compare  this  data  against  the 
expert  decisions  and  different  models  of  learning  for 
the  CIC,  tactical  decision-making  task.  Although  we 
have  collected  and  distributed  this  data,  it  has  not 
yet  been  analyzed. 

Conclusions :  Although  subjects  can  accurately  acquire 
probabilities  through  experience  with  a  tactical  decision 
making  task,  order  effects  are  still  present  without 
extended  training.  However,  with  extended  training  (i.e., 
experience  on  the  task)  order  effects  disappear.  In 
addition,  changing  situations  (such  as  from  hostile  to 
peaceful)  can  negatively  affect  decision  making,  but  has 
less  of  an  effect  than  predicted  by  normative  models.  The 
hybrid  symbolic-connectionist  cognitive  model  produced 
under  this  grant  can  predict  the  amount  of  training  needed 
to  eliminate  order  effects  and  to  adequately  prepare  a 
person  for  a  different  situation. 

Significance:  The  resulting  theory  and  model  of  probability 
acquisition  and  use  is  generally  applicable  to  a  wide  range 
of  decision-making  tasks.  Furthermore,  the  model  integrates 
existing  phenomena  regarding  the  learning  and  use  of 
probabilities,  order  effects  in  decision  making,  and  newly 
discovered  phenomena  regarding  the  relationship  between 
probability  acquisition  and  order  effects. 
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